{
 "cells": [
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "# The MIT License (MIT)\n",
    "\n",
    "# Copyright (c) 2020, NVIDIA CORPORATION.\n",
    "\n",
    "# Permission is hereby granted, free of charge, to any person obtaining a copy of\n",
    "# this software and associated documentation files (the \"Software\"), to deal in\n",
    "# the Software without restriction, including without limitation the rights to\n",
    "# use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of\n",
    "# the Software, and to permit persons to whom the Software is furnished to do so,\n",
    "# subject to the following conditions:\n",
    "\n",
    "# The above copyright notice and this permission notice shall be included in all\n",
    "# copies or substantial portions of the Software.\n",
    "\n",
    "# THE SOFTWARE IS PROVIDED \"AS IS\", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR\n",
    "# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS\n",
    "# FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR\n",
    "# COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER\n",
    "# IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN\n",
    "# CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Tutorial: Feature Engineering for Recommender Systems\n",
    "\n",
    "# 6. Scaling to Production Systems\n",
    "\n",
    "## 6.2. Introduction to NVTabular"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "With the rapid growth in scale of industry datasets, deep learning (DL) recommender models have started to gain advantages over traditional methods by capitalizing on large amounts of training data.\n",
    "\n",
    "The current challenges for training large-scale recommenders include:\n",
    "\n",
    "* **Huge datasets:** Commercial recommenders are trained on huge datasets, often several terabytes in scale.\n",
    "* **Complex data preprocessing and feature engineering pipelines:** Datasets need to be preprocessed and transformed into a form relevant to be used with DL models and frameworks. In addition, feature engineering creates an extensive set of new features from existing ones, requiring multiple iterations to arrive at an optimal solution.\n",
    "* **Input bottleneck:** Data loading, if not well optimized, can be the slowest part of the training process, leading to under-utilization of high-throughput computing devices such as GPUs.\n",
    "* **Extensive repeated experimentation:** The whole data engineering, training, and evaluation process is generally repeated many times, requiring significant time and computational resources."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**NVTabular** is a library for fast tabular data tranformation and loading, manipulating terabyte-scale datasets quickly. It provides best practices for feature engineering and preprocessing and a high-level abstraction to simplify code accelerating computation on the GPU using the RAPIDS cuDF library.\n",
    "\n",
    "<img src='https://developer.nvidia.com/blog/wp-content/uploads/2020/07/recommender-system-training-pipeline-1.png'>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Resoruces:**\n",
    "* [GitHub](https://github.com/NVIDIA/NVTabular/)\n",
    "* [GTC2020 Keynote Announcement](https://www.youtube.com/watch?v=zWe02O2po1U&feature=youtu.be)\n",
    "* [GTC2020 Session](https://on-demand.gputechconf.com/gtc/2020/s21651/)\n",
    "* [NVIDIA DevBlog](https://developer.nvidia.com/blog/accelerating-etl-for-recsys-on-gpus-with-nvtabular/)\n",
    "* [Examples](https://github.com/NVIDIA/NVTabular/tree/master/examples)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## HandsOn"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**NVTabular** has 4 main components:<br><br>\n",
    "**1. Dataset:** A *dataset* contains a list of files and iterates over the files. If necessary, it will read a file in chunks.<br>\n",
    "**2. Op:** An *Op* defines the calculation, which should be exectued. For example, an op could be to collect the mean/std for a column, fill in missing values or combine two categories.<br>\n",
    "**3. Workflow:** A *workflow* orchastrates the pipeline\n",
    "* It defines the contex, which columns are categorical, numerical or the label\n",
    "* It registers the operations (calculation) for the different column types\n",
    "* It optimizes the tasks by reordering the operations\n",
    "* It collects the required statistics for operations (e.g. the mean/std for normalization)\n",
    "* It applies the final operations to the dataset\n",
    "\n",
    "**4. Dataloader:** NVTabular provides optimized dataloader for tabular data in PyTorch and Tensorflow "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Let's build the preprocessing and feature engineering pipeline, learned in the handson tutorial, with NVTabular"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Data processing for XGBoost training"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "import nvtabular as nvt\n",
    "from nvtabular import ops"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "First, we define the paths for the training and validation dataset."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/conda/envs/nvtabular/lib/python3.7/site-packages/nvtabular/io/parquet.py:75: UserWarning: Row group size 2565426129 is bigger than requested part_size 2376558182\n",
      "  f\"Row group size {rg_byte_size_0} is bigger than requested part_size \"\n"
     ]
    }
   ],
   "source": [
    "import glob\n",
    "\n",
    "train_paths = glob.glob('../data/train.parquet')\n",
    "valid_paths = glob.glob('../data/valid.parquet')\n",
    "\n",
    "train_dataset = nvt.Dataset(train_paths, engine='parquet', part_mem_fraction=0.15)\n",
    "valid_dataset = nvt.Dataset(valid_paths, engine='parquet', part_mem_fraction=0.15)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(['../data/train.parquet'], ['../data/valid.parquet'])"
      ]
     },
     "execution_count": 3,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "train_paths, valid_paths"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We define the data schema."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [],
   "source": [
    "proc = nvt.Workflow(\n",
    "    cat_names=['product_id', 'brand', 'user_id',\n",
    "       'user_session', 'cat_0', 'cat_1', 'cat_2', 'cat_3',\n",
    "       'ts_hour', 'ts_minute', 'ts_weekday', 'ts_day', 'ts_month', 'ts_year'],\n",
    "    cont_names=['price', 'timestamp'],\n",
    "    label_name=['target']\n",
    ")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [],
   "source": [
    "proc.add_feature([\n",
    "    ops.LambdaOp(\n",
    "        op_name = 'user_id',\n",
    "        f = lambda col, gdf: col.astype(str) + '_' + gdf['user_id'].astype(str),\n",
    "        columns = ['product_id', 'brand', 'ts_hour', 'ts_minute'],\n",
    "        replace=False\n",
    "    ),\n",
    "    ops.LambdaOp(\n",
    "        op_name = 'user_id_brand',\n",
    "        f = lambda col, gdf: col.astype(str) + '_' + gdf['user_id'].astype(str) + '_' + gdf['brand'].astype(str),\n",
    "        columns = ['ts_hour', 'ts_weekday', 'cat_0', 'cat_1', 'cat_2'],\n",
    "        replace=False\n",
    "    ),\n",
    "    ops.Categorify(\n",
    "        freq_threshold=15,\n",
    "        columns = [x + '_user_id' for x in ['product_id', 'brand', 'ts_hour', 'ts_minute']] + [x + '_user_id_brand' for x in ['ts_hour', 'ts_weekday', 'cat_0', 'cat_1', 'cat_2']] + ['product_id', 'brand', 'user_id', 'user_session', 'cat_0', 'cat_1', 'cat_2', 'cat_3', 'ts_hour', 'ts_minute', 'ts_weekday', 'ts_day', 'ts_month', 'ts_year']\n",
    "    ),\n",
    "    ops.LambdaOp(\n",
    "        op_name = 'product_id',\n",
    "        f = lambda col, gdf: col.astype(str) + '_' + gdf['product_id'].astype(str),\n",
    "        columns = ['brand', 'user_id', 'cat_0'],\n",
    "        replace=False\n",
    "    ),\n",
    "    ops.JoinGroupby(\n",
    "        cont_names=[]\n",
    "    ),\n",
    "    ops.TargetEncoding(\n",
    "        cat_groups = ['brand', 'user_id', 'product_id', 'cat_2', ['ts_weekday','ts_day']],\n",
    "        cont_target= 'target',\n",
    "        kfold=5,\n",
    "        fold_seed=42,\n",
    "        p_smooth=20,\n",
    "    )\n",
    "])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [],
   "source": [
    "!rm -r ./output_nvt_train/\n",
    "!mkdir ./output_nvt_train/\n",
    "\n",
    "!rm -r ./output_nvt_valid/\n",
    "!mkdir ./output_nvt_valid/\n",
    "\n",
    "proc.apply(train_dataset,  \n",
    "           output_path='./output_nvt_train/'\n",
    "          )\n",
    "\n",
    "proc.apply(valid_dataset,  \n",
    "           output_path='./output_nvt_valid/',\n",
    "           record_stats=False\n",
    "          )"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We added following data operations:\n",
    "<li> Combine Categories\n",
    "<li> Categorify Categories\n",
    "<li> Count Encoding (JoinGroupBy)\n",
    "<li> Target Encoding\n",
    "<br><br>\n",
    "More features/ops are continously added to NVTabular and the other ops will follow soon.\n",
    "We load the train and valid dataset and run the additional features.\n",
    "    <br><br>\n",
    "First, we load the data, again."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "{'status': 'ok', 'restart': True}"
      ]
     },
     "execution_count": 7,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "import IPython\n",
    "\n",
    "app = IPython.Application.instance()\n",
    "app.kernel.do_shutdown(True)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [],
   "source": [
    "import cudf\n",
    "import pandas as pd\n",
    "import glob \n",
    "\n",
    "train_paths = glob.glob('./output_nvt_train/*.parquet')\n",
    "valid_paths = glob.glob('./output_nvt_valid/*.parquet')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [],
   "source": [
    "train = cudf.concat([cudf.read_parquet(x) for x in train_paths])\n",
    "valid = cudf.concat([cudf.read_parquet(x) for x in valid_paths])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>price</th>\n",
       "      <th>timestamp</th>\n",
       "      <th>product_id</th>\n",
       "      <th>brand</th>\n",
       "      <th>user_id</th>\n",
       "      <th>cat_0</th>\n",
       "      <th>cat_1</th>\n",
       "      <th>cat_2</th>\n",
       "      <th>cat_3</th>\n",
       "      <th>ts_hour</th>\n",
       "      <th>...</th>\n",
       "      <th>cat_1_user_id_brand</th>\n",
       "      <th>cat_2_user_id_brand</th>\n",
       "      <th>brand_product_id_count</th>\n",
       "      <th>user_id_product_id_count</th>\n",
       "      <th>cat_0_product_id_count</th>\n",
       "      <th>TE_brand_target</th>\n",
       "      <th>TE_user_id_target</th>\n",
       "      <th>TE_product_id_target</th>\n",
       "      <th>TE_cat_2_target</th>\n",
       "      <th>TE_ts_weekday_ts_day_target</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>49.91</td>\n",
       "      <td>2020-03-01 00:00:59</td>\n",
       "      <td>10955</td>\n",
       "      <td>2914</td>\n",
       "      <td>55813</td>\n",
       "      <td>8</td>\n",
       "      <td>49</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>...</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>156</td>\n",
       "      <td>&lt;NA&gt;</td>\n",
       "      <td>156</td>\n",
       "      <td>0.257370</td>\n",
       "      <td>0.239448</td>\n",
       "      <td>0.399650</td>\n",
       "      <td>0.322939</td>\n",
       "      <td>0.389486</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>397.10</td>\n",
       "      <td>2020-03-01 00:01:20</td>\n",
       "      <td>31</td>\n",
       "      <td>155</td>\n",
       "      <td>0</td>\n",
       "      <td>6</td>\n",
       "      <td>51</td>\n",
       "      <td>41</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>...</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>141423</td>\n",
       "      <td>84792</td>\n",
       "      <td>103293</td>\n",
       "      <td>0.421481</td>\n",
       "      <td>0.338761</td>\n",
       "      <td>0.521025</td>\n",
       "      <td>0.459643</td>\n",
       "      <td>0.389486</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>823.70</td>\n",
       "      <td>2020-03-01 00:01:52</td>\n",
       "      <td>100</td>\n",
       "      <td>155</td>\n",
       "      <td>0</td>\n",
       "      <td>6</td>\n",
       "      <td>51</td>\n",
       "      <td>41</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>...</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>7568</td>\n",
       "      <td>4856</td>\n",
       "      <td>3500</td>\n",
       "      <td>0.421481</td>\n",
       "      <td>0.338761</td>\n",
       "      <td>0.328590</td>\n",
       "      <td>0.459643</td>\n",
       "      <td>0.389486</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>422.15</td>\n",
       "      <td>2020-03-01 00:02:14</td>\n",
       "      <td>23020</td>\n",
       "      <td>2195</td>\n",
       "      <td>0</td>\n",
       "      <td>12</td>\n",
       "      <td>53</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>...</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>25</td>\n",
       "      <td>24</td>\n",
       "      <td>22</td>\n",
       "      <td>0.106112</td>\n",
       "      <td>0.338761</td>\n",
       "      <td>0.251966</td>\n",
       "      <td>0.322939</td>\n",
       "      <td>0.389486</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>69.24</td>\n",
       "      <td>2020-03-01 00:02:15</td>\n",
       "      <td>5632</td>\n",
       "      <td>176</td>\n",
       "      <td>0</td>\n",
       "      <td>3</td>\n",
       "      <td>20</td>\n",
       "      <td>82</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>...</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>50</td>\n",
       "      <td>43</td>\n",
       "      <td>50</td>\n",
       "      <td>0.323030</td>\n",
       "      <td>0.338761</td>\n",
       "      <td>0.347693</td>\n",
       "      <td>0.350999</td>\n",
       "      <td>0.389486</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>...</th>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2461714</th>\n",
       "      <td>2.65</td>\n",
       "      <td>2020-03-31 23:57:47</td>\n",
       "      <td>28311</td>\n",
       "      <td>566</td>\n",
       "      <td>0</td>\n",
       "      <td>3</td>\n",
       "      <td>35</td>\n",
       "      <td>42</td>\n",
       "      <td>0</td>\n",
       "      <td>24</td>\n",
       "      <td>...</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>66</td>\n",
       "      <td>47</td>\n",
       "      <td>80</td>\n",
       "      <td>0.166703</td>\n",
       "      <td>0.338761</td>\n",
       "      <td>0.221259</td>\n",
       "      <td>0.396022</td>\n",
       "      <td>0.374029</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2461715</th>\n",
       "      <td>234.96</td>\n",
       "      <td>2020-03-31 23:58:19</td>\n",
       "      <td>41306</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>24</td>\n",
       "      <td>...</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>51</td>\n",
       "      <td>46</td>\n",
       "      <td>52</td>\n",
       "      <td>0.301020</td>\n",
       "      <td>0.338761</td>\n",
       "      <td>0.338034</td>\n",
       "      <td>0.322939</td>\n",
       "      <td>0.374029</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2461716</th>\n",
       "      <td>223.92</td>\n",
       "      <td>2020-03-31 23:58:20</td>\n",
       "      <td>5336</td>\n",
       "      <td>2281</td>\n",
       "      <td>23311</td>\n",
       "      <td>3</td>\n",
       "      <td>20</td>\n",
       "      <td>82</td>\n",
       "      <td>0</td>\n",
       "      <td>24</td>\n",
       "      <td>...</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1013</td>\n",
       "      <td>1</td>\n",
       "      <td>1013</td>\n",
       "      <td>0.439617</td>\n",
       "      <td>0.549198</td>\n",
       "      <td>0.349795</td>\n",
       "      <td>0.350999</td>\n",
       "      <td>0.374029</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2461717</th>\n",
       "      <td>100.36</td>\n",
       "      <td>2020-03-31 23:59:19</td>\n",
       "      <td>42652</td>\n",
       "      <td>2771</td>\n",
       "      <td>0</td>\n",
       "      <td>3</td>\n",
       "      <td>20</td>\n",
       "      <td>82</td>\n",
       "      <td>0</td>\n",
       "      <td>24</td>\n",
       "      <td>...</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>55</td>\n",
       "      <td>44</td>\n",
       "      <td>55</td>\n",
       "      <td>0.318733</td>\n",
       "      <td>0.338761</td>\n",
       "      <td>0.564513</td>\n",
       "      <td>0.350999</td>\n",
       "      <td>0.374029</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2461718</th>\n",
       "      <td>319.41</td>\n",
       "      <td>2020-03-31 23:59:27</td>\n",
       "      <td>42366</td>\n",
       "      <td>2281</td>\n",
       "      <td>0</td>\n",
       "      <td>6</td>\n",
       "      <td>51</td>\n",
       "      <td>41</td>\n",
       "      <td>0</td>\n",
       "      <td>24</td>\n",
       "      <td>...</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>15300</td>\n",
       "      <td>9802</td>\n",
       "      <td>15301</td>\n",
       "      <td>0.439617</td>\n",
       "      <td>0.338761</td>\n",
       "      <td>0.525706</td>\n",
       "      <td>0.459643</td>\n",
       "      <td>0.374029</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>2461719 rows × 33 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "          price            timestamp  product_id  brand  user_id  cat_0  \\\n",
       "0         49.91  2020-03-01 00:00:59       10955   2914    55813      8   \n",
       "1        397.10  2020-03-01 00:01:20          31    155        0      6   \n",
       "2        823.70  2020-03-01 00:01:52         100    155        0      6   \n",
       "3        422.15  2020-03-01 00:02:14       23020   2195        0     12   \n",
       "4         69.24  2020-03-01 00:02:15        5632    176        0      3   \n",
       "...         ...                  ...         ...    ...      ...    ...   \n",
       "2461714    2.65  2020-03-31 23:57:47       28311    566        0      3   \n",
       "2461715  234.96  2020-03-31 23:58:19       41306      0        0      0   \n",
       "2461716  223.92  2020-03-31 23:58:20        5336   2281    23311      3   \n",
       "2461717  100.36  2020-03-31 23:59:19       42652   2771        0      3   \n",
       "2461718  319.41  2020-03-31 23:59:27       42366   2281        0      6   \n",
       "\n",
       "         cat_1  cat_2  cat_3  ts_hour  ...  cat_1_user_id_brand  \\\n",
       "0           49      0      0        1  ...                    0   \n",
       "1           51     41      0        1  ...                    0   \n",
       "2           51     41      0        1  ...                    0   \n",
       "3           53      0      0        1  ...                    0   \n",
       "4           20     82      0        1  ...                    0   \n",
       "...        ...    ...    ...      ...  ...                  ...   \n",
       "2461714     35     42      0       24  ...                    0   \n",
       "2461715      0      0      0       24  ...                    0   \n",
       "2461716     20     82      0       24  ...                    0   \n",
       "2461717     20     82      0       24  ...                    0   \n",
       "2461718     51     41      0       24  ...                    0   \n",
       "\n",
       "         cat_2_user_id_brand  brand_product_id_count  \\\n",
       "0                          0                     156   \n",
       "1                          0                  141423   \n",
       "2                          0                    7568   \n",
       "3                          0                      25   \n",
       "4                          0                      50   \n",
       "...                      ...                     ...   \n",
       "2461714                    0                      66   \n",
       "2461715                    0                      51   \n",
       "2461716                    0                    1013   \n",
       "2461717                    0                      55   \n",
       "2461718                    0                   15300   \n",
       "\n",
       "         user_id_product_id_count cat_0_product_id_count  TE_brand_target  \\\n",
       "0                            <NA>                    156         0.257370   \n",
       "1                           84792                 103293         0.421481   \n",
       "2                            4856                   3500         0.421481   \n",
       "3                              24                     22         0.106112   \n",
       "4                              43                     50         0.323030   \n",
       "...                           ...                    ...              ...   \n",
       "2461714                        47                     80         0.166703   \n",
       "2461715                        46                     52         0.301020   \n",
       "2461716                         1                   1013         0.439617   \n",
       "2461717                        44                     55         0.318733   \n",
       "2461718                      9802                  15301         0.439617   \n",
       "\n",
       "         TE_user_id_target  TE_product_id_target  TE_cat_2_target  \\\n",
       "0                 0.239448              0.399650         0.322939   \n",
       "1                 0.338761              0.521025         0.459643   \n",
       "2                 0.338761              0.328590         0.459643   \n",
       "3                 0.338761              0.251966         0.322939   \n",
       "4                 0.338761              0.347693         0.350999   \n",
       "...                    ...                   ...              ...   \n",
       "2461714           0.338761              0.221259         0.396022   \n",
       "2461715           0.338761              0.338034         0.322939   \n",
       "2461716           0.549198              0.349795         0.350999   \n",
       "2461717           0.338761              0.564513         0.350999   \n",
       "2461718           0.338761              0.525706         0.459643   \n",
       "\n",
       "         TE_ts_weekday_ts_day_target  \n",
       "0                           0.389486  \n",
       "1                           0.389486  \n",
       "2                           0.389486  \n",
       "3                           0.389486  \n",
       "4                           0.389486  \n",
       "...                              ...  \n",
       "2461714                     0.374029  \n",
       "2461715                     0.374029  \n",
       "2461716                     0.374029  \n",
       "2461717                     0.374029  \n",
       "2461718                     0.374029  \n",
       "\n",
       "[2461719 rows x 33 columns]"
      ]
     },
     "execution_count": 4,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "train.drop(['user_session', 'brand_product_id', 'user_id_product_id', 'cat_0_product_id'], inplace=True)\n",
    "valid.drop(['user_session', 'brand_product_id', 'user_id_product_id', 'cat_0_product_id'], inplace=True)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We define the functions for additional feature engineering."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [],
   "source": [
    "import cupy\n",
    "\n",
    "# TARGET ENCODE WITH KFOLD\n",
    "\n",
    "def target_encode2(train, valid, col, target='target', kfold=5, smooth=20, verbose=True):\n",
    "    \"\"\"\n",
    "        train:  train dataset\n",
    "        valid:  validation dataset\n",
    "        col:   column which will be encoded (in the example RESOURCE)\n",
    "        target: target column which will be used to calculate the statistic\n",
    "    \"\"\"\n",
    "    \n",
    "    # We assume that the train dataset is shuffled\n",
    "    train['kfold'] = ((train.index) % kfold)\n",
    "    # We keep the original order as cudf merge will not preserve the original order\n",
    "    train['org_sorting'] = cupy.arange(len(train), dtype=\"int32\")\n",
    "    # We create the output column, we fill with 0\n",
    "    col_name = '_'.join(col)+'_'+str(smooth)\n",
    "    train['TE_' + col_name] = 0.\n",
    "    for i in range(kfold):\n",
    "        ###################################\n",
    "        # filter for out of fold\n",
    "        # calculate the mean/counts per group category\n",
    "        # calculate the global mean for the oof\n",
    "        # calculate the smoothed TE\n",
    "        # merge it to the original dataframe\n",
    "        ###################################\n",
    "        \n",
    "        df_tmp = train[train['kfold']!=i]\n",
    "        mn = df_tmp[target].mean()\n",
    "        df_tmp = df_tmp[col + [target]].groupby(col).agg(['mean', 'count']).reset_index()\n",
    "        df_tmp.columns = col + ['mean', 'count']\n",
    "        df_tmp['TE_tmp'] = ((df_tmp['mean']*df_tmp['count'])+(mn*smooth)) / (df_tmp['count']+smooth)\n",
    "        df_tmp_m = train[col + ['kfold', 'org_sorting', 'TE_' + col_name]].merge(df_tmp, how='left', left_on=col, right_on=col).sort_values('org_sorting')\n",
    "        df_tmp_m.loc[df_tmp_m['kfold']==i, 'TE_' + col_name] = df_tmp_m.loc[df_tmp_m['kfold']==i, 'TE_tmp']\n",
    "        train['TE_' + col_name] = df_tmp_m['TE_' + col_name].fillna(mn).values\n",
    "\n",
    "    \n",
    "    ###################################\n",
    "    # calculate the mean/counts per group for the full training dataset\n",
    "    # calculate the global mean\n",
    "    # calculate the smoothed TE\n",
    "    # merge it to the original dataframe\n",
    "    # drop all temp columns\n",
    "    ###################################    \n",
    "    \n",
    "    df_tmp = train[col + [target]].groupby(col).agg(['mean', 'count']).reset_index()\n",
    "    mn = train[target].mean()\n",
    "    df_tmp.columns = col + ['mean', 'count']\n",
    "    df_tmp['TE_tmp'] = ((df_tmp['mean']*df_tmp['count'])+(mn*smooth)) / (df_tmp['count']+smooth)\n",
    "    valid['org_sorting'] = cupy.arange(len(valid), dtype=\"int32\")\n",
    "    df_tmp_m = valid[col + ['org_sorting']].merge(df_tmp, how='left', left_on=col, right_on=col).sort_values('org_sorting')\n",
    "    valid['TE_' + col_name] = df_tmp_m['TE_tmp'].fillna(mn).values\n",
    "    \n",
    "    valid = valid.drop('org_sorting', axis=1)\n",
    "    train = train.drop('kfold', axis=1)\n",
    "    train = train.drop('org_sorting', axis=1)\n",
    "    return (train, valid, 'TE_'+col_name)\n",
    "\n",
    "def group_binning(df, valid, q_list = [0.1, 0.25, 0.5, 0.75, 0.9]):\n",
    "    df['price_bin'] = -1\n",
    "    valid['price_bin'] = -1\n",
    "    \n",
    "    for i, q_value in enumerate(q_list):\n",
    "        print(q_value)\n",
    "        q = df[['cat_012', 'price']].groupby(['cat_012']).quantile(q_value)\n",
    "        q = q.reset_index()\n",
    "        q.columns = ['cat_012', 'price' + str(q_value)]\n",
    "        df = df.merge(q, how='left', on='cat_012')\n",
    "        valid = valid.merge(q, how='left', on='cat_012')\n",
    "        if i == 0:\n",
    "            df.loc[df['price']<=df['price' + str(q_value)], 'price_bin'] = i\n",
    "            valid.loc[valid['price']<=valid['price' + str(q_value)], 'price_bin'] = i\n",
    "        else:\n",
    "            df.loc[(df['price']>df['price' + str(q_list[i-1])]) & (df['price']<=df['price' + str(q_value)]), 'price_bin'] = i\n",
    "            valid.loc[(valid['price']>valid['price' + str(q_list[i-1])]) & (valid['price']<=valid['price' + str(q_value)]), 'price_bin'] = i\n",
    "        if i>=2:\n",
    "            df.drop(['price' + str(q_list[i-2])], axis=1, inplace=True)\n",
    "            valid.drop(['price' + str(q_list[i-2])], axis=1, inplace=True)\n",
    "            \n",
    "    df.loc[df['price']>df['price' + str(q_value)], 'price_bin'] = i+1\n",
    "    df.drop(['price' + str(q_list[i-1])], axis=1, inplace=True)\n",
    "    df.drop(['price' + str(q_list[i])], axis=1, inplace=True)\n",
    "    \n",
    "    valid.loc[valid['price']>valid['price' + str(q_value)], 'price_bin'] = i+1\n",
    "    valid.drop(['price' + str(q_list[i-1])], axis=1, inplace=True)\n",
    "    valid.drop(['price' + str(q_list[i])], axis=1, inplace=True)\n",
    "    \n",
    "def rolling_window(train, valid, col, offset):\n",
    "    df = cudf.concat([train, valid]) \n",
    "    data_window = df[[col, 'date', 'target']].groupby([col, 'date']).agg(['count', 'sum']).reset_index()\n",
    "    data_window.columns = [col, 'date', 'count', 'sum']\n",
    "    data_window.index = data_window['date']\n",
    "    \n",
    "    data_window_roll = data_window[[col, 'count', 'sum']].groupby([col]).rolling(offset).sum().drop(col, axis=1)\n",
    "    data_window_roll = data_window_roll.reset_index()\n",
    "    data_window_roll.columns = [col, 'date', col + '_count_' + offset, col + '_sum_' + offset]\n",
    "    data_window_roll[[col + '_count_' + offset, col + '_sum_' + offset]] = data_window_roll[[col + '_count_' + offset, col + '_sum_' + offset]].shift(1)\n",
    "    data_window_roll.loc[data_window_roll[col]!=data_window_roll[col].shift(1), [col + '_count_' + offset, col + '_sum_' + offset]] = 0\n",
    "    data_window_roll[col + '_avg_' + offset] = (data_window_roll[col + '_sum_' + offset]/data_window_roll[col + '_count_' + offset]).fillna(-1)\n",
    "    df = df.merge(data_window_roll, how='left', on=[col, 'date'])\n",
    "    train = df[df['ts_month']!=3]\n",
    "    valid = df[df['ts_month']==3]\n",
    "    return(train, valid)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We bin the price and target encode the price bins."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [],
   "source": [
    "train['cat_012'] = train['cat_0'].astype(str) + '_' + train['cat_1'].astype(str) + '_' + train['cat_2'].astype(str)\n",
    "valid['cat_012'] = valid['cat_0'].astype(str) + '_' + valid['cat_1'].astype(str) + '_' + valid['cat_2'].astype(str)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "0.1\n",
      "0.25\n",
      "0.5\n",
      "0.75\n",
      "0.9\n"
     ]
    }
   ],
   "source": [
    "group_binning(train, valid)\n",
    "train, valid, name = target_encode2(train, valid, ['price_bin'], 'target', smooth=20)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We create the time window features."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/conda/envs/nvtabular/lib/python3.7/site-packages/cudf/core/column/column.py:1396: UserWarning: Date32 values are not yet supported so this will be typecast to a Date64 value\n",
      "  UserWarning,\n"
     ]
    }
   ],
   "source": [
    "train['date'] = cudf.from_pandas(pd.to_datetime(train['timestamp'].to_pandas()).dt.date)\n",
    "valid['date'] = cudf.from_pandas(pd.to_datetime(valid['timestamp'].to_pandas()).dt.date)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "Index(['price', 'timestamp', 'product_id', 'brand', 'user_id', 'cat_0',\n",
       "       'cat_1', 'cat_2', 'cat_3', 'ts_hour', 'ts_minute', 'ts_weekday',\n",
       "       'ts_day', 'ts_month', 'ts_year', 'target', 'product_id_user_id',\n",
       "       'brand_user_id', 'ts_hour_user_id', 'ts_minute_user_id',\n",
       "       'ts_hour_user_id_brand', 'ts_weekday_user_id_brand',\n",
       "       'cat_0_user_id_brand', 'cat_1_user_id_brand', 'cat_2_user_id_brand',\n",
       "       'brand_product_id_count', 'user_id_product_id_count',\n",
       "       'cat_0_product_id_count', 'TE_brand_target', 'TE_user_id_target',\n",
       "       'TE_product_id_target', 'TE_cat_2_target',\n",
       "       'TE_ts_weekday_ts_day_target', 'cat_012', 'price_bin',\n",
       "       'TE_price_bin_20', 'date'],\n",
       "      dtype='object')"
      ]
     },
     "execution_count": 9,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "train.columns"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [],
   "source": [
    "train['product_user'] = train['product_id'].astype(str) + '_' + train['user_id'].astype(str) + '_' + train['cat_2'].astype(str)\n",
    "valid['product_user'] = valid['product_id'].astype(str) + '_' + valid['user_id'].astype(str) + '_' + valid['cat_2'].astype(str)\n",
    "# LABEL ENCODE CATEGORIES\n",
    "comb = cudf.concat([train,valid],ignore_index=True)\n",
    "for c in ['product_user']: \n",
    "    tmp,code = comb[c].factorize()\n",
    "    train[c] = tmp[:len(train)].values\n",
    "    valid[c] = tmp[len(train):].values"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "Index(['price', 'timestamp', 'product_id', 'brand', 'user_id', 'cat_0',\n",
       "       'cat_1', 'cat_2', 'cat_3', 'ts_hour', 'ts_minute', 'ts_weekday',\n",
       "       'ts_day', 'ts_month', 'ts_year', 'target', 'product_id_user_id',\n",
       "       'brand_user_id', 'ts_hour_user_id', 'ts_minute_user_id',\n",
       "       'ts_hour_user_id_brand', 'ts_weekday_user_id_brand',\n",
       "       'cat_0_user_id_brand', 'cat_1_user_id_brand', 'cat_2_user_id_brand',\n",
       "       'brand_product_id_count', 'user_id_product_id_count',\n",
       "       'cat_0_product_id_count', 'TE_brand_target', 'TE_user_id_target',\n",
       "       'TE_product_id_target', 'TE_cat_2_target',\n",
       "       'TE_ts_weekday_ts_day_target', 'cat_012', 'price_bin',\n",
       "       'TE_price_bin_20', 'date', 'product_user'],\n",
       "      dtype='object')"
      ]
     },
     "execution_count": 11,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "train.columns"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We drop the unused columns"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>price</th>\n",
       "      <th>product_id</th>\n",
       "      <th>brand</th>\n",
       "      <th>user_id</th>\n",
       "      <th>cat_0</th>\n",
       "      <th>cat_1</th>\n",
       "      <th>cat_2</th>\n",
       "      <th>cat_3</th>\n",
       "      <th>ts_hour</th>\n",
       "      <th>ts_minute</th>\n",
       "      <th>...</th>\n",
       "      <th>brand_product_id_count</th>\n",
       "      <th>user_id_product_id_count</th>\n",
       "      <th>cat_0_product_id_count</th>\n",
       "      <th>TE_brand_target</th>\n",
       "      <th>TE_user_id_target</th>\n",
       "      <th>TE_product_id_target</th>\n",
       "      <th>TE_cat_2_target</th>\n",
       "      <th>TE_ts_weekday_ts_day_target</th>\n",
       "      <th>TE_price_bin_20</th>\n",
       "      <th>product_user</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>49.91</td>\n",
       "      <td>10955</td>\n",
       "      <td>2914</td>\n",
       "      <td>55813</td>\n",
       "      <td>8</td>\n",
       "      <td>49</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>...</td>\n",
       "      <td>156</td>\n",
       "      <td>&lt;NA&gt;</td>\n",
       "      <td>156</td>\n",
       "      <td>0.257370</td>\n",
       "      <td>0.239448</td>\n",
       "      <td>0.399650</td>\n",
       "      <td>0.322939</td>\n",
       "      <td>0.389486</td>\n",
       "      <td>0.366924</td>\n",
       "      <td>107700</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>397.10</td>\n",
       "      <td>31</td>\n",
       "      <td>155</td>\n",
       "      <td>0</td>\n",
       "      <td>6</td>\n",
       "      <td>51</td>\n",
       "      <td>41</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>2</td>\n",
       "      <td>...</td>\n",
       "      <td>141423</td>\n",
       "      <td>84792</td>\n",
       "      <td>103293</td>\n",
       "      <td>0.421481</td>\n",
       "      <td>0.338761</td>\n",
       "      <td>0.521025</td>\n",
       "      <td>0.459643</td>\n",
       "      <td>0.389486</td>\n",
       "      <td>0.366924</td>\n",
       "      <td>678289</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>823.70</td>\n",
       "      <td>100</td>\n",
       "      <td>155</td>\n",
       "      <td>0</td>\n",
       "      <td>6</td>\n",
       "      <td>51</td>\n",
       "      <td>41</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>2</td>\n",
       "      <td>...</td>\n",
       "      <td>7568</td>\n",
       "      <td>4856</td>\n",
       "      <td>3500</td>\n",
       "      <td>0.421481</td>\n",
       "      <td>0.338761</td>\n",
       "      <td>0.328590</td>\n",
       "      <td>0.459643</td>\n",
       "      <td>0.389486</td>\n",
       "      <td>0.366924</td>\n",
       "      <td>80568</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>422.15</td>\n",
       "      <td>23020</td>\n",
       "      <td>2195</td>\n",
       "      <td>0</td>\n",
       "      <td>12</td>\n",
       "      <td>53</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>3</td>\n",
       "      <td>...</td>\n",
       "      <td>25</td>\n",
       "      <td>24</td>\n",
       "      <td>22</td>\n",
       "      <td>0.106112</td>\n",
       "      <td>0.338761</td>\n",
       "      <td>0.251966</td>\n",
       "      <td>0.322939</td>\n",
       "      <td>0.389486</td>\n",
       "      <td>0.366924</td>\n",
       "      <td>411417</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>69.24</td>\n",
       "      <td>5632</td>\n",
       "      <td>176</td>\n",
       "      <td>0</td>\n",
       "      <td>3</td>\n",
       "      <td>20</td>\n",
       "      <td>82</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>3</td>\n",
       "      <td>...</td>\n",
       "      <td>50</td>\n",
       "      <td>43</td>\n",
       "      <td>50</td>\n",
       "      <td>0.323030</td>\n",
       "      <td>0.338761</td>\n",
       "      <td>0.347693</td>\n",
       "      <td>0.350999</td>\n",
       "      <td>0.389486</td>\n",
       "      <td>0.366924</td>\n",
       "      <td>1116076</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>...</th>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2461714</th>\n",
       "      <td>2.65</td>\n",
       "      <td>28311</td>\n",
       "      <td>566</td>\n",
       "      <td>0</td>\n",
       "      <td>3</td>\n",
       "      <td>35</td>\n",
       "      <td>42</td>\n",
       "      <td>0</td>\n",
       "      <td>24</td>\n",
       "      <td>58</td>\n",
       "      <td>...</td>\n",
       "      <td>66</td>\n",
       "      <td>47</td>\n",
       "      <td>80</td>\n",
       "      <td>0.166703</td>\n",
       "      <td>0.338761</td>\n",
       "      <td>0.221259</td>\n",
       "      <td>0.396022</td>\n",
       "      <td>0.374029</td>\n",
       "      <td>0.366924</td>\n",
       "      <td>552770</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2461715</th>\n",
       "      <td>234.96</td>\n",
       "      <td>41306</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>24</td>\n",
       "      <td>59</td>\n",
       "      <td>...</td>\n",
       "      <td>51</td>\n",
       "      <td>46</td>\n",
       "      <td>52</td>\n",
       "      <td>0.301020</td>\n",
       "      <td>0.338761</td>\n",
       "      <td>0.338034</td>\n",
       "      <td>0.322939</td>\n",
       "      <td>0.374029</td>\n",
       "      <td>0.366924</td>\n",
       "      <td>901890</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2461716</th>\n",
       "      <td>223.92</td>\n",
       "      <td>5336</td>\n",
       "      <td>2281</td>\n",
       "      <td>23311</td>\n",
       "      <td>3</td>\n",
       "      <td>20</td>\n",
       "      <td>82</td>\n",
       "      <td>0</td>\n",
       "      <td>24</td>\n",
       "      <td>59</td>\n",
       "      <td>...</td>\n",
       "      <td>1013</td>\n",
       "      <td>1</td>\n",
       "      <td>1013</td>\n",
       "      <td>0.439617</td>\n",
       "      <td>0.549198</td>\n",
       "      <td>0.349795</td>\n",
       "      <td>0.350999</td>\n",
       "      <td>0.374029</td>\n",
       "      <td>0.366924</td>\n",
       "      <td>1088926</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2461717</th>\n",
       "      <td>100.36</td>\n",
       "      <td>42652</td>\n",
       "      <td>2771</td>\n",
       "      <td>0</td>\n",
       "      <td>3</td>\n",
       "      <td>20</td>\n",
       "      <td>82</td>\n",
       "      <td>0</td>\n",
       "      <td>24</td>\n",
       "      <td>60</td>\n",
       "      <td>...</td>\n",
       "      <td>55</td>\n",
       "      <td>44</td>\n",
       "      <td>55</td>\n",
       "      <td>0.318733</td>\n",
       "      <td>0.338761</td>\n",
       "      <td>0.564513</td>\n",
       "      <td>0.350999</td>\n",
       "      <td>0.374029</td>\n",
       "      <td>0.366924</td>\n",
       "      <td>940873</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2461718</th>\n",
       "      <td>319.41</td>\n",
       "      <td>42366</td>\n",
       "      <td>2281</td>\n",
       "      <td>0</td>\n",
       "      <td>6</td>\n",
       "      <td>51</td>\n",
       "      <td>41</td>\n",
       "      <td>0</td>\n",
       "      <td>24</td>\n",
       "      <td>60</td>\n",
       "      <td>...</td>\n",
       "      <td>15300</td>\n",
       "      <td>9802</td>\n",
       "      <td>15301</td>\n",
       "      <td>0.439617</td>\n",
       "      <td>0.338761</td>\n",
       "      <td>0.525706</td>\n",
       "      <td>0.459643</td>\n",
       "      <td>0.374029</td>\n",
       "      <td>0.366924</td>\n",
       "      <td>931768</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>2461719 rows × 34 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "          price  product_id  brand  user_id  cat_0  cat_1  cat_2  cat_3  \\\n",
       "0         49.91       10955   2914    55813      8     49      0      0   \n",
       "1        397.10          31    155        0      6     51     41      0   \n",
       "2        823.70         100    155        0      6     51     41      0   \n",
       "3        422.15       23020   2195        0     12     53      0      0   \n",
       "4         69.24        5632    176        0      3     20     82      0   \n",
       "...         ...         ...    ...      ...    ...    ...    ...    ...   \n",
       "2461714    2.65       28311    566        0      3     35     42      0   \n",
       "2461715  234.96       41306      0        0      0      0      0      0   \n",
       "2461716  223.92        5336   2281    23311      3     20     82      0   \n",
       "2461717  100.36       42652   2771        0      3     20     82      0   \n",
       "2461718  319.41       42366   2281        0      6     51     41      0   \n",
       "\n",
       "         ts_hour  ts_minute  ...  brand_product_id_count  \\\n",
       "0              1          1  ...                     156   \n",
       "1              1          2  ...                  141423   \n",
       "2              1          2  ...                    7568   \n",
       "3              1          3  ...                      25   \n",
       "4              1          3  ...                      50   \n",
       "...          ...        ...  ...                     ...   \n",
       "2461714       24         58  ...                      66   \n",
       "2461715       24         59  ...                      51   \n",
       "2461716       24         59  ...                    1013   \n",
       "2461717       24         60  ...                      55   \n",
       "2461718       24         60  ...                   15300   \n",
       "\n",
       "         user_id_product_id_count cat_0_product_id_count  TE_brand_target  \\\n",
       "0                            <NA>                    156         0.257370   \n",
       "1                           84792                 103293         0.421481   \n",
       "2                            4856                   3500         0.421481   \n",
       "3                              24                     22         0.106112   \n",
       "4                              43                     50         0.323030   \n",
       "...                           ...                    ...              ...   \n",
       "2461714                        47                     80         0.166703   \n",
       "2461715                        46                     52         0.301020   \n",
       "2461716                         1                   1013         0.439617   \n",
       "2461717                        44                     55         0.318733   \n",
       "2461718                      9802                  15301         0.439617   \n",
       "\n",
       "         TE_user_id_target  TE_product_id_target  TE_cat_2_target  \\\n",
       "0                 0.239448              0.399650         0.322939   \n",
       "1                 0.338761              0.521025         0.459643   \n",
       "2                 0.338761              0.328590         0.459643   \n",
       "3                 0.338761              0.251966         0.322939   \n",
       "4                 0.338761              0.347693         0.350999   \n",
       "...                    ...                   ...              ...   \n",
       "2461714           0.338761              0.221259         0.396022   \n",
       "2461715           0.338761              0.338034         0.322939   \n",
       "2461716           0.549198              0.349795         0.350999   \n",
       "2461717           0.338761              0.564513         0.350999   \n",
       "2461718           0.338761              0.525706         0.459643   \n",
       "\n",
       "         TE_ts_weekday_ts_day_target  TE_price_bin_20  product_user  \n",
       "0                           0.389486         0.366924        107700  \n",
       "1                           0.389486         0.366924        678289  \n",
       "2                           0.389486         0.366924         80568  \n",
       "3                           0.389486         0.366924        411417  \n",
       "4                           0.389486         0.366924       1116076  \n",
       "...                              ...              ...           ...  \n",
       "2461714                     0.374029         0.366924        552770  \n",
       "2461715                     0.374029         0.366924        901890  \n",
       "2461716                     0.374029         0.366924       1088926  \n",
       "2461717                     0.374029         0.366924        940873  \n",
       "2461718                     0.374029         0.366924        931768  \n",
       "\n",
       "[2461719 rows x 34 columns]"
      ]
     },
     "execution_count": 12,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "train.drop(['timestamp', 'cat_012', 'price_bin', 'date'] , inplace=True)\n",
    "valid.drop(['timestamp', 'cat_012', 'price_bin', 'date'] , inplace=True)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [],
   "source": [
    "#train, valid = rolling_window(train, valid, 'product_user', '1D')\n",
    "#train, valid = rolling_window(train, valid, 'product_user', '7D')\n",
    "#train, valid = rolling_window(train, valid, 'product_user', '14D')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We save the new dataframes to disk."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [],
   "source": [
    "train.to_parquet('train_fe.parquet')\n",
    "valid.to_parquet('valid_fe.parquet')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Train XGBoost"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We train a XGBoost classifier"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "{'status': 'ok', 'restart': True}"
      ]
     },
     "execution_count": 15,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "import IPython\n",
    "\n",
    "app = IPython.Application.instance()\n",
    "app.kernel.do_shutdown(True)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [],
   "source": [
    "import cudf"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [],
   "source": [
    "train = cudf.read_parquet('train_fe.parquet')\n",
    "valid = cudf.read_parquet('valid_fe.parquet')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "Index(['price', 'product_id', 'brand', 'user_id', 'cat_0', 'cat_1', 'cat_2',\n",
       "       'cat_3', 'ts_hour', 'ts_minute', 'ts_weekday', 'ts_day', 'ts_month',\n",
       "       'ts_year', 'target', 'product_id_user_id', 'brand_user_id',\n",
       "       'ts_hour_user_id', 'ts_minute_user_id', 'ts_hour_user_id_brand',\n",
       "       'ts_weekday_user_id_brand', 'cat_0_user_id_brand',\n",
       "       'cat_1_user_id_brand', 'cat_2_user_id_brand', 'brand_product_id_count',\n",
       "       'user_id_product_id_count', 'cat_0_product_id_count', 'TE_brand_target',\n",
       "       'TE_user_id_target', 'TE_product_id_target', 'TE_cat_2_target',\n",
       "       'TE_ts_weekday_ts_day_target', 'TE_price_bin_20', 'product_user'],\n",
       "      dtype='object')"
      ]
     },
     "execution_count": 5,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "train.columns"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [],
   "source": [
    "features = [\n",
    "    'price', \n",
    "    'product_id', \n",
    "    'brand', \n",
    "    'user_id', \n",
    "    'cat_0', \n",
    "    'cat_1', \n",
    "    'cat_2',\n",
    "    'cat_3', \n",
    "    'ts_hour', \n",
    "    'ts_minute', \n",
    "    'ts_weekday', \n",
    "    'ts_day', \n",
    "    'ts_month',\n",
    "    'ts_year', \n",
    "    'product_id_user_id', \n",
    "    'brand_user_id',\n",
    "    'ts_hour_user_id', \n",
    "    'ts_minute_user_id', \n",
    "    'ts_hour_user_id_brand',\n",
    "    'ts_weekday_user_id_brand', \n",
    "    'cat_0_user_id_brand',\n",
    "    'cat_1_user_id_brand', \n",
    "    'cat_2_user_id_brand', \n",
    "    'brand_product_id_count',\n",
    "    'user_id_product_id_count', \n",
    "    'cat_0_product_id_count', \n",
    "    'TE_brand_target',\n",
    "    'TE_user_id_target', \n",
    "    'TE_product_id_target', \n",
    "    'TE_cat_2_target',\n",
    "    'TE_ts_weekday_ts_day_target', \n",
    "    'TE_price_bin_20'\n",
    "]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [],
   "source": [
    "xgb_parms = { \n",
    "    'max_depth':12, \n",
    "    'learning_rate':0.02, \n",
    "    'subsample':0.4,\n",
    "    'colsample_bytree':0.4, \n",
    "    #'eval_metric':'logloss',\n",
    "    'eval_metric':'auc',\n",
    "    'objective':'binary:logistic',\n",
    "    'tree_method':'gpu_hist',\n",
    "    'seed': 123\n",
    "}"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[0]\ttrain-auc:0.69630\tvalid-auc:0.59201\n",
      "Multiple eval metrics have been passed: 'valid-auc' will be used for early stopping.\n",
      "\n",
      "Will train until valid-auc hasn't improved in 50 rounds.\n",
      "[25]\ttrain-auc:0.76714\tvalid-auc:0.64098\n",
      "[50]\ttrain-auc:0.76994\tvalid-auc:0.64318\n",
      "[75]\ttrain-auc:0.77490\tvalid-auc:0.64397\n",
      "[100]\ttrain-auc:0.77682\tvalid-auc:0.64449\n",
      "[125]\ttrain-auc:0.77918\tvalid-auc:0.64530\n",
      "[150]\ttrain-auc:0.78142\tvalid-auc:0.64558\n",
      "[175]\ttrain-auc:0.78271\tvalid-auc:0.64567\n",
      "Stopping. Best iteration:\n",
      "[137]\ttrain-auc:0.78063\tvalid-auc:0.64599\n",
      "\n"
     ]
    }
   ],
   "source": [
    "import xgboost as xgb\n",
    "\n",
    "\n",
    "NROUND = 1000\n",
    "ESR = 50\n",
    "VERBOSE_EVAL = 25\n",
    "dtrain = xgb.DMatrix(data=train[features],label=train.target)\n",
    "dvalid = xgb.DMatrix(data=valid[features],label=valid.target)\n",
    "                \n",
    "model = xgb.train(xgb_parms, \n",
    "                  dtrain=dtrain,\n",
    "                  evals=[(dtrain,'train'),(dvalid,'valid')],\n",
    "                  num_boost_round=NROUND,\n",
    "                  early_stopping_rounds=ESR,\n",
    "                  verbose_eval=VERBOSE_EVAL) "
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.8"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}
